On the curved exponential family in the Stochastic Approximation Expectation Maximization Algorithm

The Expectation-Maximization Algorithm (EM) is a widely used method allowing to estimate the maximum likelihood of models involving latent variables. When Expectation step cannot be computed easily, one can use stochastic versions EM such as Stochastic Approximation EM. This algorithm, however, has drawback require joint belong curved exponential family. To overcome this problem, [16] introduced rewriting model which “exponentializes” it by considering parameter an additional variable following Normal distribution centered on newly defined parameters and with fixed variance. new exponentialized now belongs Although often used, there no guarantee that estimated mean close initial model. In paper, we quantify error done in estimation while instead one. By verifying those results example, see trade-off must made between speed convergence tolerated error. Finally, propose algorithm better reasonable computation time reduce bias.